NATO STO

In the commercial industry, AI models have achieved notable successes in automated scene understanding. From enhancing security through intelligent surveillance systems to advancing automated driving, these models play a key role in diverse applications. However, the successes of industry have not yet been leveraged to address the military problem because relevant military data needed for training models does not exist in sufficient quantity and is typically not available or of interest to industry leaders. It is very hard to collect sufficient representative samples of military relevant data, for example when it comes to using other imagery such as thermal infra-red, or for scenarios in inaccessible terrain. For many military use cases such training datasets are not available. Often acquisition and labelling of such datasets is too expensive, and in the case of very rare events, or during system development, impossible as the data cannot be recorded. As emphasized by the SET-272 RTG ‘Using AI for Automated Scene Understanding’, a solution to the lack of access to sufficient real training data is to simulate this data. Potentially this would lead to high quality AI models, since with simulations we can, in principle, generate large representative datasets for any scenario of interest. In many fields NATO already has sophisticated models of the application, scenarios, phenomena, apparatus and sensors; exploiting this knowledge should provide a good path to generate high quality AI models. However, despite promising attempts to use simulation for training data, the members of ET-134 identified that currently AI-models trained on simulated data do not necessarily perform well on real world data. To bridge this gap, this RTG aims to study simulation variation, image realism, and underlying physics to obtain simulated data that enables strong AI model development. Another goal is to determine what AI (training) methods work well when using synthetic data.